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ABSTB&CT 

Tbe Inforaation Exchange Procedures (lEPy cost studi 
proiect of the National Center for Higher Education Hanageaent 
Systeas is described and its applicability to six aaior res«earcb 
universities (HHOI is assessed in this pilot study. The lEP enables 
pe«r institutions to coapare inforaation about their resources, 
activities, and educational cutcoaes. The technical findings of six 
universities {University of Colorado, University of Illinois, 
Oniversity of Kansas, Purdue University, State university of New ¥ork 
at Stony Brock, Oniversity of «ashingtoa> based on their experience 
in iapleaenting lEP are discussed. Eight stens that each uni versity 
folloifed in iapleaenting the costing procedure are described. K 
discussion of the aaior findings covers the areas of faculty activity 
analysis, departmental research and the loint-product probleo, MRU 
accounting procedures, aodif ica tions in the f ull-cost -procedures of 
lEP, discipline and student- prograa data, and aodif icatior.s to 
NCHBBiS's costing and data aanageaent software. The strengths and 
veaknesscs of lEP for SHOs are analyzed. It is concluded that 
although the data were not substantively or technically cooparable, 
advances bad fceen aade through the pilot study toward achieving 
jtechnical coaparability between institutions. The reed for 
substantive coaparability in the lEP is noted and discussed, 
■potential applications of cost lata for each pilot-test institution 
are described. (SF) 
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Preface 



An original charge to NCHEMS was to develop a set of procedures and defini- 
tions that would enable peer institutions to compare information about their 
resources, activities, and educational outcomes. To fulfill this responsibility, 
NCHEMS developed in the early 1970s what eventually became known as the 
Information Exchange Procedures (lEP). Though lEP was generally accepted 
by colleges and universities, one sector of higher education—the major 
research universities— expressed reservations as to the applicability and 
validity of the procedures to their situation. Thus in early 1975, several of 
them petitioned the NCHEMS Board of Directors to reexamine certain 
portions of lEP from their perspective. 

A task force representing the major research universities was subse- 
quently appointed by the Board. Two working groups were formed— one to 
focus on the costing methods contained in lEP and the other on ahernative 
approaches to information exchange. The first group, subsequently known 
as the Experimental Application and Analysis* Subgroup, conducted an 
active pilot test of the costing portions of lEP to determine their relevance to 
a major research university. This work is documented ' i two reporcs: 

• Evaluation of the lEP Costing Procedures: A Pilot Study by Six Major 
Research Universities (1979). A report that summarizes the technical 
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findings of six major research universities based upon their experience 
in implementing NCHfiMS Information Exchange Procedures. Partici- 
pating in the study were the University of Colorado, University of 
Illinois, University of Kansas, Purdue University, State University of 
New York at Stony Brook, and the University of Washington, 
• Tedmical Diary of the Major Research Uni'uersitks' Pilot Test (1979). A 
step-by-stcp commentary on the impieiLcntation and analysis of the 
NCHEMS cost-study procedures. Modifications to adapt the procedures 
to major research universities are included as part of the advisory-group 
recommendations. 

The second group, known as the Measures and Definitions Subgroup, 
examined alternative approaches to information exchange among major 
research universities. Its work, largely conceptual in nature, was developed 
by representatives from public and piivate universities, including Stanford 
University, State University of New York at Stony Brook, University of 
California at Los Angeles, University of Michigan, and University of 
Rochester. Their report is entitled Information Exchange Procedures for Major 
Research Universities: Alternative Conceptual Approaches (1979). 

Together, these three documents constitute the final report of the 
NCHEMS Major Research Universities Task Force. NCHEMS is indebted 
to the participants in this project for their contribution of time and energy. 
While substantive conclusions were not reached in ail aspect? of the study, 
significant progress was made in exploring the issues sur aunding informa- 
tion exchange among major research universities and, in some instances, in 
suggesting tentative solutions to the problems. We publish these reports in 
the hope that they will help other universities that want to undertake similar 
comparative studies. 

A. Ray Chamberlain 
Chairperson, Board of Directors 

Ben Lawrence 
Executive Director 

Jim Topping 
Project Director 
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Why This Study Was Undertaken 



Colleges and universities compete with one another for scarce resources. 
Within institutions, academic departments and administrative units engage in 
similar competition. Despite this competition, no firm benchmarks exist for 
assessing the demand for losources in higher education. Though o le might 
expect the allocation of resources to reflect the expected yield of products or 
services, the outputs of higher education are .intangible, and there are no 
widely accepted conventions for measuring their value. Despite these limita- 
tions, comparisons both among and within institutions are common and have 
become increasingly influential in resource-allccatifjn decisions. Thus in 1971, 
the National Center for Higher Education Management Systems (NCHEMS) 
initiated the Information Exchange Procedures (lEP) project to: 

• Support institutional identification, acquisition and use of information 
necessary to carry out institutional comparative analysis, particularly in 

» resource allocation 

• Improve the quality and quantity of data that an institution can bring to 
bear on internal planning and management problems 

N'oTP Thii chapter is bawd on "Mator Research I'nivcrsitics Experimental Application and Ani'!v,is Subgroup 
Siatcrocni of Purf>o»e ' (1977), "Introductory Notes by the Advisory Committee" (Bacchctti et a! forthcoming), 
and "Apples are Never OranKe<. The Comparability QucMion Rcviiiied" (Harris 197H) 

1 
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• Assist institutions in the preparation and presentation of uati to agencies 
and^ individuals external to the campus [NCHEMS 1975, p 77] 

These objectives and the expected uses of lEP reflect several assumptions 
basic to the project. The project was based on the conviction th^t the 
availability of better information will lead to improvements in institutional 
planning and management, but that simply providing more information to 
dccisiorunakers will not necessarily do so. Those developing lEP assum::d 
that comparative analysis would help executives formulate alternatives in 
planning and management decisions. They also assumed that a set of selected 
historical data would be a \iseful starting point for comparative analysis and 
that sufficiently compatible data can be collected across institutions by estab- 
lishing 3 fixed structure, definitions, and a set of procedures. Previously a 
widely held assumption was that the same structure, definitions, and pro- 
cedures could be used for any educational institution. 

As the results of the lEP project were disseminated, it became apparent 
that some states intended to use lEP to make intrastate cost comparisons 
among publicly supported colleges and universities. These plans to make 
comparisons across populations that ranged from community colleges to major 
research imiversities (MRU) aroused considerable concern among pi'anners 
and administrators at major research universities. They believed that the 
underlying asgimiptions and design of lEP were inadequate to capture and 
display the salient chai acteristics of a major research university and that 
comparisons that failed to recognize these important differences in institu- 
tional character and mission might be invahd and could adversely affect the 
ftinding of complex research-intensive universities. 

In studying the distinctions among major research universities, two im- 
portant characteristics were identified. 

Perhaps the single most management-relevant characteristic of MRUs 
is . . . [that] institutional purposes are largely elaborations on the duties 
of the faculty, succinctly bat accurately stated in the excerpt from the 
Statutes of the University of Cambridge: 

The University duties of faculty members arc to devote themselves co the 
ai^vanccmcnt of knowledge in their subjects, to give the students instruction 
in those subjects, and to promote the interests of the University as a place 
of education, learning, and research. 
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This characteristic of the university as an organization means that uni- 
versities arc as much settings within which education, research [and 
public service] go on as they are organizations responsible for doin'^ 
teaching, research [and public sen'icc], and certifying learning. As 
organizations, they have as large a stake in the intellectual independence 
of faculty and students as they do in cultural transmission, curricuiar 
coherence, the effective use of resources, and social progress. 

A second distinctive feature of MRUs is the unification of [the] 
teaching, research [and public service] objecil 'es within individuals and 
organizational units. So close is the coupling of these functions . . . that 
facts or information can only >>e properly understood in a larger context. 
The importance of this methodological stricture is often overlooked. 
The natural tendency of information systems is atomistic; they aim at 
reducing a whole to its smallest parts to display alternative interpreta- 
tions. This natural tendency is not always legitimate. If two parts or 
aspects of something must both be present in order to represent it truly, 
then no intellectual purpose is served by reducing this union to its 
separate parts. In analyzing universities, this problem has come to be 
called the problem of jointness: When a library book seives lx)rh a 
research and a teaching objective, or when a professor in conducting 
research is also teaching a student, the functions are unified, are joint, 
and the frame of reference must marry, not divorce them in order for in- 
furmation about library books or how professors spend their tmie to 
convey meaning. [Bacchetti et al forthcoming] 

To respond to expressions of thtse concerns by representatives of several 
MRUs, in 1975, the NCHEMS Board established the MRU-IEP project 
and appointed a task force to examine the f( 'owing questions: 

1 . Given institutional differences in accounting, organization, and so 
forth, can data be collected from a ^ .up of major research universities 
in accordance with NCHEMS Information Exchange Procedures unacr 
controlled pilot-test cond...ons so that the resulting information is 
reasonably comparable? 

2. Docs IE? have the capability to deal with the interactions among in- 
struction, research, and public service characteristic of major research 
universities? 

3. Do existing information exchange procedures adequately account tor 
different sources of funding in complex, research-oriented universuies? 



4. Do existing informafion exchange procedures adequately define direct 
and indirect costs to permit comparisons among major research uni- 
versities? 

5. Do the costing procedures recommeiided by the existing information 
exchange procedures adequately reflect the consumption of support 
services in major research universities? 

6. How similar are major research universities to one another? To other 
postsecondary institutions? 

The task force was also charged with developing approaches that would meet 
the needs of MRUs if it found lEP inadequate. 

At the first meeting, task-force members differed on the approach to their 
assignment, borne favored testing the validity of lEP for major research 
universities by means of a controlled experiment. Others believed that the 
conceptual framework of lEP should be reexamined and, if necessary, re- 
defined. As a compromise, it was decided that the task fcr-^e should proceed 
as two subgroups. 

The first group, the Experimental Application and Analysis Subgroup, 
adopted three guidelines for conducting its work: 

• Data would not he forced to demonsirate comparability among schools, 
disciplines, or programs 

• Modifications and additions to lEF would be made as needed, with the 
agreement of the group 

• The experiment would be terminated at any time that the group con- 
cluded that lEP was inapplicable to major research universities 

The purpose of these guidelines was to assure an objective, honest test of 
lEP rather than an effort to make lEP work regardless of the compromises. 

The second approach was pursued by the Measures and Definitions 
Subgroup. Its work resulted in a draft manual entitled Information Exchange 
Procedures for Major Research Universities: Alternative Conceptual Approaches 
circulated for review in late 1977, % 

The work of the Experimental Application and Analysis Subgroup has 
now been completed. The remainder of this report describes how the test 
was conducted, presents the major findings and conclusions, and discusses 
the implications of lEP for comparative analysis within and among major 
research universities. 
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Organization of the Study 



Implementation of the Information Exchange Procedures (lEP) cost study at 
any coilege or university is a complex task, often involving the collection and 
analysis of data m various formats from several different institutional offices. 
The task is farther complicated when one attempts to collect data in com- 
patible formats frpm several institutions. To overcome these problems, 
NCHEMS designed a general set of costing procedures to accommodate a 
wide range of institutional data-collection practices. While NCHEMS costing 
procedures do not guarantee data comparability, they provide a common 
framework for conducting comparative cost studies. The Major Research 
University-Information Exchange Procedures (MRU-IEP) study attempted 
to determine if the NCHEMS costing procedures and associated software 
can support comparative cost studies among major research universities. 

The objeciive of the MRU-IEF cost study was to develop programmatic, 
full-cost data by discipline and course level and by student major and student 
level. It is not the purpose of this report to explicate all of the general lEP 
procedures and guidelines for producing programmatic full-cost data; those 
procedures are covered in other NCHEMS technical reports, specifically 
Procedures for Determining Historical Full Costs (1977). Instead, this chapter 
will briefly describe the sequence of steps that each university followed in 
implementing the NCHEMS costing procedures. Chapter 3 will then discuss 
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the problems that were encountered in implementin^h^e costing procedures 
in major research universities (MRUs) and will present the technical fmdings 
and recommendations made by the pilot-test group for modifying the existing 
procedures. These procedural modifications are elaborated on in the Technical 
Diary of the MRU Pilot Test (Topping forthcoming). 



Eight Steps of the Cost Study 

The implementation process for the MRU-IEP -ost study was organized 
into the following eight steps (see figure 1). 

Step I: Inventory of Stu(knt Programs and Academic Disciplines. Each insti- 
tution was asked to prepare an inventory of its student programs and discipline 
offerings using the four-digit codes employed by the National Center for 
Education Statistics in its annual Higher Education General Information 
Survey (HEGIS). The purpose was to identify discipline and student-program 
groupings that would permit meaningful comparisons of dat^uring the later 
steps of the study. 

Step 2: Collection and Analysis of Student-Registration Data. In this step, 
an Instructional Work Load Matrix (IWLM) was developed for each insti- 
tution. Constructed from s.udent-registration data, this matrix quantifies the 
relationships between the academic departments offering the course work 
and the declared majors of the students taking the copses. The IWLMs were 
later used as the transition matrices to produce programmatic cost information. 

Step 3: Collection and Analysis of Faculty- Activity Data. Most institutions 
participating in the pilot test already had faculty-activity reporting systems. 
The major task in this step was to review each institution's faculty-activity 
reporting system and, as a group, determine a set of faculty activities common 
to all the institution?. These common activities included the primary programs 
of instruction, organized research, public service, and academic-support 
functions necessary for representing an MRU. Faculty data, including both 
activity and coinpensation information, were needed to determine how the 
expendhures of an academic unit were to be distributed to lEP activity 
centers. 

Step 4: Cross-Check of Errors between Faculty and Student Data. To relate 
faculty data to student data, both sets had to be coded to an identical activity 
structure. Frequent mismatches among disciplines and course levels occurred. 
The purpose of this step was to resolve mismatches between the two data sets 
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before proceeding with the analysis and Limultaneously to resolve otb-ir 
errors in the two sets- 

Step 5: Callection and Analysis of Ins^nutional Expenditure Data and Pro- 
ductian of Modified Direct-Cost Data. In step 5, each institution's curre.it 
funds expenditure data were segregated by fund category, reconciled, and 
crossed over lo the lEP activity structure. Again, the purpose was to ensure 
that like activities (expenditures in this case) were categorized in a similar 
manner to permit comparisons across institutions. An additional series of 
departmental allocations was added to the procedures to adjust for differing 
faculty-activity surveys. Finally, it was important that each instirution recon- 
ciled its total current funds expenditures to its audited financial statements. 

Step 6: Calculation and Analysis of Modified Direct Unit Costs for Academic 
Disciplines and Student Programs. A unit cost is simply the total cost of an 
activity or service divided by the number of units of that activity or service 
produced within a given period. Expressing costs in leims of units of activity 
Tacilitates comparisons— with a predetermined cost, a cost of past activities, 
or the cost of a similar unit of activity in another organization. In this study, 
the agreed-upon unit of activity for instruction was the semester credit hour 
(except for doctoral dissertation, for which each enrollment was counted as a 
single unit). Steps 1 and 2 facilitated the collection and analysis of the credit- 
hour information, that is, the denominator of the unit-cost equation; steps 3, 
4, and 5 were concerned with the collection and analysis of the cost data, or 
the numerator of the equation. Step 6 was the tlrst attempt to merge these 
data files to produce unii-cosi data for each of the two-digit HEGIS 
discipline categories and for each of the course levels within those HEGIS 
categories. Similarly, through the use of the Instructional Work Load Matrix 
produced in step 2, modified direct unit costs were produced for each student 
program and each student level within those student programs. 

Step 7: Collection and Analysis of Assignable Square Feet and Allocation of 
Indirect Costs. Betbre full unit costs could be produced, each institution needed 
to perform a series of allocations distributing their indirect-cost pools to those 
cost centers previously designated as fmal-cost objectives. In lEP, the fmal- 
cost objectives fail principally within three primary programs—instruction, 
research, and public service— but also include some of the student-support 
services normally treated as auxiliary services, such as dormitories, food 
services, intercollegiate athletics, and all independent operations. All other 
support services were considered indirect and were therefore allocated to the 
final-cost objectives. As part of this step, assignable-square-feet data were 
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collected and analysed to facilitate the allocation of plant operation and 
maintenance. 

Step 8: Calculation and Analysis of Full Unit Costs for Academic Disciplines 
and Student Programs, Once full costs were determined for each discipline 
and course level, unit costs were calculated by dividing those totals by student 
credit hours generated at each course Itw iL By multiplying the full unit costs 
through the Instructional Work Load Matrix, unit costs were also produced 
for each student major and student level. These figures were then convened 
to cost per PTE student by multiplying the unit cost figures by a previously 
agreed-upon standard. 

The Pilot Tcss 

The pilot test began in March 1976 and continued for two and one-half 
years. It focused on summarized data from the 1975-76 fiscal year. The pilot- 
test group initially consisted of five university campuses: 

• University of Colorado at Boulder 

• The University of Kansas at Lawrence 

• Purdue University at West Lafayette 

• University of Illinois at Urbana-Champaign 
Slate University of New York at Stony Brook 

A sixth university, the University of Washington, joined the group midway 
through the pilot test.. With the exception of data from the University of 
Washington, most data dealing with the medical schools and the teaching 
hospitals were excluded to avoid complicating the analyses. 

Figure 2 indicates the approximate time devoted to each step of the pilot 
test. One should not conclude from examining figure 2 that it .takes two and 
one-half years to implement the procedures contained in MRU-IEP. Imple- 
mentation time for the University of Washington, which joined the pilot test 
at a later date^ was in the range of six to eight months. The pilot test was 
substantially lengthened in accordance with its guiding philosophy,^ th^the 
effort was not simply six universities independently implementing lEP and 

i It js fhc t.olle<.nve opinjon o\ the p>k>t-!t'st ^fuup ihaf the tinic |HfKHi required U>r ^uccessp'c comparative cost 
itudics among MRTs could he nubi^fantuliv reduted if the pfoMcms <!.scu!<»scd in chapter 3 were ^aiislactonly 
rc»oK'cd pnor fo implementation of the piu4.ednirs 
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then comparing the final results. Rather, the pilot-test institutions decided at 
their initial meeting that following each step of the implementation process, 
they would carefully review the data to evaluate the degree of comparability 
among the six institutions. If they determined that the data in the step being 
examined were no: comparable, hnplementation would halt until a satisfactory 
solution to the problem could be found. The purpose of this step-by-step 
implementation and evaluation process was to ensure the validity of the 
investigation. For each step, the group completely documented the problems 
encountered, analyses performed, and conclusions reached. As part of this 
documentation, each university submitted a protocol statement describing 
problems that it encountered in the data-collection process and methods for 
solving them. The complete documentation of the pilot test, including the 
protocol statements submitted by the pilot-test schools, is contained in a 
separate book (Topping forthcoming). 



Cost of the Cost Study 

Cost estimates for the two and one-half year implementation period 
ranged from $30,000 to $62,000 per institution. Roughly one-third of those 
costs was attributable to data processing, the remainder to staff salaries, 
benefits, and travel. Cost estimates varied among the six universities for 
several reasons. Paramount among these was the degree of difTiculty of adapt- 
ing existing institutional data bases to the Information Exchange Procedures. 
Another was the amount of experience of university staff in conducting 
comparative cost studies, and a third was the amount of time senior-level 
administrators devoted to the project. 

In addition to the institutional costs, NCHEMS contributed about 
$200,000 to support the pilot-test effort. NCHEMS staff processed a major 
portion of the universities' data, developed and tested additional software for 
the MRU modifications to the procedures, analyzed data, and produced 
reports tc facilitate comparisons of the data. The balance of NCHEMS efforts 
was in general support of the project, which included staff support of the 
study group, drafting and publication of the documents, and staff and study- 
group travel. 

A university that seeks to replicate the MRU-IEP cost study should not 
expect that the cost estimates of this pilot test will reflect its own costs. The 
nature of the pilot test tended to expand the timi frame, thereby increasing the 
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costs to the institutions. However, NCHEMS provided a substantial amount 
of lechnicai and general support, thereby decreasing each mstitution's total 
cost. It is the opinion of the six pilot-test institutions that an MRU attempt' ig 
to replicate this cost study without technical assistance and with no experience 
with the lEP costing procedures could double and perhaps triple the imple- 
mentation costs reported in this pilot test. 
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Technical Findings of the Study 



Testing the Information Exchange Procedures (lEP) soon revealed that sub- 
stantial modificalions or additions would be needed to address the questions 
raised in the charge to the study grouR. This chapter discusses the major 
technical findings of the pilot test, listed in the order of their significance to 
the fmal outcome— establishing compaiable sets of cost data for the six 
universities. 



Faculty Activity Analysis 

A critical element of this kind of cost study is the Faculty Activity Analysis 
(FAA). Facuhy-activity data are used to allocate faculty compensation costs 
to various activities. Subsequently the same allocation percentages are us'^d 
to allocate academic-department support costs to activities. Faculty-effort 
data, which historically have been a major problem in cost studies, not sur- 
prisingly presented significant problems in this study as well. 

Certain technical problems were found in using existing faculty-activity 
information of participating institutions. This study was conduced retro- 
spectively on facuhy-activity data previously collected and analyzed by the 
pilot-test institutions. Many of these problems could have been avoided if the 



13 



participating institutions had agreed in advance on a single faculty-activity 
survey instrument and had administered it uniformly. Technical problems 
included: 

• Not ai! institutions had complete FAAs for all the terms or semesters 
being exammed and had to derive information from periods available. 
In particular, summer data were not always available. 

• Populations included in the FAAs varied. Some institutions surveyed 
only ranked instructional faculty. Others surveyed broader pools that 
included all instmction and research faculty, administrators, extramuially 
funded research staff, assistants, and others with academic titles. 

• DifTei ences in definition or assignment of full-time equivalents (PTEs) 
particularly aflected reporting for graduate assistants who had FTE 
values for similar appointments ranging from 0.25 to 0.50. (This partic- 
ularly affected intermediate reports on such matters as number of service 
months and productivity ratios. All costs were ultimately reported, but 
assignment to activity or level could be influenced by these diflerences.) 

• The number of reporting categories varied from 9 to more than 15. The 
group suggests a small but uniform set for future cost studies to include 
the following: 

—Instruction and related activities 

—General fund or restricted fund research in institutes and research 
^ centers 

—Other separately budgeted research 
—Departmental research and professional development 
— Academic^ student, and institutional support 
—Public service 

—Cooperative extension services 

It appears that at least this number r: 2ctivities are needed to report the 
multiple faculty activities in major /esearch universities. Additional data 
needed for internal use should be treated as subsets of these aggregated cate- 
gories. To reiterate, if major research universities were to agree prospectively 
on the FAAs to be used, many of these technical problems could be resolved. 

In addition to the technical problems, two major conceptual problems 
arose in working with the FAAs. The first, regarding reporting methods, is 
described below. The second, concerning departmental research, is treated 
in the following section. One of the most significant conceptual differences 
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was having faculty actiyiiy reported by instiiuiional assignments (in contrasi 
to self-reporting of professional time across a broader grid of activities). A 
hypothetical example of applying the two reporting methods to the activities 
of a single individual illustrates the ditTcrences. (In this example, we assume 
no separately budgeted research or service activity.) 



Activity 


Assigsmient 


Sclf-RcportUig 


Instruction of courses 


Instruction 


Instruction 


Institutes and research 


VT A 
IN/i 


NJA 


centers 






Separately budgeted research 


NA 


NA 


Depairtmenial research and 


Instruction 


Research 


professional development 






Academic, student, and 


Instruction 


Institutional administration, 


institutional support 




Student services or 






Academic administration 


Public service 


Instruction 


Public service 


Cooperative extension 


NA 
\ / 


NA 



The underlined activities illustrate the diflerences in reporting methods. 
Ordinarily, under the assignment method, a full-time faculty member is 
expected to do departmental research and professional development, participate 
in departmental and institutional committees, advise students, and perform 
public service as part of his assigned activity. Only special assignments, such 
as departmental administrative duties, new course development, or an excep- 
tional amount of research or public service, would be recognized in the FAA 
report prepared for the faculty members. In contrast, under the self-reporting 
method, the faculty member's estimate of how his professional time is spent is 
used to distribute faculty effort to program categories regarded as part of the 
so called normal assignment and to the other activities shown in the example. 

Two participating institutions used the assignment method, and four used 
seU-reportmg. The difTerences in reported activity w':i e so significant that the 
study team decided to treat all the faculty activitv as though it had been on an 
assignment basis in oidci to carry out the other tests. Therefore Mod,ficd 
Direct Costs were produced, in which faculty activity reported undc-,- depart- 
mental research, course and curriculum development, and student advising 
were prorytcd back as instructional expense. Academic iidministratusn was 
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prorated to instruction^ research, and public service, The use of Modified 
Direa Cost produced a greater level of comparability among the pilot -test 
institutions. 



Departmental Research and the Joint-Product Problem 

A major objective of the MRU-IEF project was to devise ways to account 
adequately for the joint production of instruction, research, and public service 
in which major research, doctoral-granting institutions engage^ The Ad Hoc 
Committee that identified the problems that Information Exchange Pro- 
cedures presented for major research universities summarized the joint- 
product problem: 

Joint Products: lEP require^ that departmental activities and theret^re 
departmental costs be disaggregated into instruction, research, an;l 
public-service components. To a large extent, this is now being done on 
the basis of primary intent. The interrelationships c .hese three costmg 
components have never been satisfactorily examined for the purposes of 
costing. This problem is particularly acute for major research univei- 
sities. [NCHEMS 1975b, p. 126] 

One question addressed in the study was: Do existing lEP proc^dLre:^ 
adequately treat the costing of activities that produce joint products? {f not. 
what modifications are necessary? The answer is: No, lEP lacks procedures 
to aid institutions in joint costing. Furthermore, the study group does not * 
know of any other techniques now available that adequately address this 
problem. The joint product problem is less significant when major research 
universities (MRUs) are making comparisons with each other than with 
institutions having other roles and missions. Faculty in MRUs are generally 
expected to do research or other scholarly activity and public service as part of 
their institutional assignment. Tangible outputs of the scholarly and service 
work arc evident in such forms as papers, books, objects of art, artistic per 
formances, and patients seen. Although there is tangible evidence of this ac- 
tivity, it is difficult to cost and even more difficult to calculate benefits. The 
major problem with joint products arises when MRU costs are compared 
with institutions having diiTcrent roles and missions. 
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A related aspect of the joint-product problem is departmental research. 
The manner in which departmental-research costs are attributed to the dif- 
ferent levels pf instruction can have significant effects on the reported costs. 
The MRU study group discussed several methods used in ailoca^ ng depart- 
mental research and could not agree on a theoretical basis for accepting any 
one of them. This study group arbitrarily decided to assign departmental 
research costs to upper division and graduate-level costs in proportion to the 
faculty compensation reported at those levels. The assumption was that the 
benefit to students of increased scholarly competence of the faculty flowed 
primarily to the upper-division majors and graduate students in the discipline. 
In addition, since much lower-division teaching in MRUs is done by graduate 
students whose stipends are based on their instruction responsibilities and 
not their research, it seemed inappropriate to a'i'xate departmental research 
costs of ranked facuhy to a pool that included a significant amount of graduate 
student compensation. Since only MRUs participated in the study, there was 
no formal opportunity to test the impact of this allocation scheme on institu- 
tions with diflferent missions. 



MRlj Accounting Procedures 

A major MRU concern about the existing lEP was that information is 
not difTcrentiated by source of funds. Large amounts of institutional activity 
at MRUs are funded through contra-- 's and grants from outside sponsors, 
including significant reimbursements for indirect costs associated with these 
activities. MRUs engage in substantial amounts of auxiliary-enterprise 
activities as well. A major change in the basic lEP procedures therefore was 
to gather all expense data by the following funding sources: 

• General funds, primarily from state appropriations and student tuitions 
(or fees charged for instructional activity) 

• Indirect-cost recoveries 

• Restricted funds from various grants, contracts, and gifts whose use is 
determined by the grantor 

• Auxiliary funds derived from user charges 
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By producing four sets of lEP cost data (on** *br each source of funds), it #as 
possible toldcntify the difTerent mix of activities supported by the four funding 
sources. This information, which is useful in comparing MRUs, is even more 
significant if MRU costs are to be compared with costs of other types of insti- 
tutions that have markedly differeni funding patterns. 

A further change in lEP procedures was to gather and retain, through 
much of the analysis, information by expenditure type. The objects of expen- 
diture used wtTt academic salaries, L^cademic fringe benefits, nonacademic 
salaries, nonacademic fringe bcnefitij, and all other current fund expenses. For 
more detailed studies, the study group suggests additional separate categories 
for graduate-assistant salaries and fringe benefits, including fee remissions. 
However, for aggregate comparisons, the study group felt that a two-part 
breakdown consisting of academic compensation (salaries and fringe benefits) 
and all other expenses would b^ sufBcient. 

TTic expenditure information was generated from institutional accounting 
iccords or outside sources and reconciled to the published financial reports. 
Even though the institution's financial reports conform to generally accepted 
accounting principles, man- ^gnificant differences in financial reporting 
must be addressed in studies i^fthis kind. The following adjustments, to the 
financial data were necessary: 

• Some states pay certain fringe-benefit costs centrally that are not reported 
in the institutional financial statements. These costs were added. 

• Many public MRUs are part of a multicampus system. Therefore it was 
necessary to add the costs of central services performed for the campuses 
and to delete costs of services performed at the MRU campus for other 
campuses in the system. 

• Some institutions budget and account for indirect-cost-recovery expen- 
ditures in a separate fund source. Other institutions treat indirect-cosl 
reco%'ery as an unrestricted general fund income and do not differentiate 
between general anJ indirect-cost-recovery expenditures. In the latter 
case, indirect-cost-recovery expenditures were prorated by the cost- 
recovery formula through which they were generated, 

• Some institutions treated service centers such as motor pools, print 
shops, and warehouses with accounting techniques that recorded expense 
botL to the user and to the service center. These duplications were elim- 
inated and the cost assigned to the user account. 
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• Some institutions regarded such cost centers as health services or residence 
hails as pan of the g^a^al fund activity. These institutions reassigned 
these expenditures to auxiliaries. ' 

• Special techniques were required to distribute faculty and staff fringe 
benefits from central accounts to various cost centers. This included 
treating fee remissions of graduate assistants as a fringe-benefit expense . 

Eac|> institution provided reconciliation statements to show the impact of 
^tfi^se adjustments on the annual-report control totals and protocol statements 
to explain the reasons for the adjustments. One adjustment that was not 
resolved related to computers or other costly equipment. Some institutions 
\ included an annual share of capital cost in their charges to users. Others had 
I piirchased machines, and therefore no current capital costs appeared in their 
• charges. Any ftiture study should adopt conventions for resolving this matter. 
The adjustments made in this cost study to the annual-report fmancial 
data were significant. Any future study should pay careful attention to the 
need for adjustments such rs those discussed in this section. 



Modifications in the Full-Cost Procedures of lEP 

Several important modifications were made in the full-costing procedures 
of lEP, including: 

• A major change in lEP allocation procedures was the introduction of 
the Modified Direct Cost step, in which faculty-activity costs initially 
reported under the self-reporting tnethod were reclassified (see p. 14). 

• Most of the svpport-cost allocations were based on modified direct costs. 

• Capital costs, for the most part, were excluded, because major cost dif- 
ferences could be introduced into the study that were solely a function 
of when a particular facility was constructed or of local construction 
costs. However, data on amounts of space available by space type were 
gathered and exchanged. 

• The six institutions varied in the extent to which support costs such as 
physical plant and administration were already allocated as part ot the 
institutional accounting policies. This was particularly true in regard to 
auxiliaries. Each institution provided specific allocation-decision rules 
to adjust for these variations. 



ERIC 



19 



• lEP doti» not readily jsfivide for allocating acadtmic-adminislfation 
cost centers, such as college deans' offices, back to the disciplines. Thus 
it was necessary to make separate allocations to assign college costs to 
disciplines before making campus-wide allocations of central costs. 

• Originally a step-down procedure^ was considered for allocating certain 
support costs for each of the four sources of funds. After testing the data 
of a single institution, it was concluded that a one-step allocation was 
sufficiently precise. 



DitcipSine and Student-Program Data 

Since a primary use of lEP is to develop discipline and student-program 
costs, agreement was necessary on the coding of course and student data. All 
the institutions reported their 1975-76 student registration information using 
the four-digit HEGIS coding. When the data were compiled, wide variation 
existed in the way in which the four-digit HEGIS codes had been assigned to 
disciplines and, to a lesser extent, to programs (see table 1). For example, 
institutions tended to use the HEGIS codes that best fit the departmental 
structures that also served as their cost centers. Furthermore, some institutions 
used the general division heading (such as 0400) instead of the first discipline 
heading ^0401) to report the same kind of activity. Even after this kmd of 
divergent reporting was corrected, the institutions still found substantial 
diversity at the disaggregated four-digit level. 

A comparison by four-digit codes of the disciplines and programs of the 
pilot-test universities showed that more than 50 percent occurred at only one 
university and could not therefore be compared^ about 20 percent occurred 
at only two universities; and roughly 30 percent occurred among comparison 
groups of three or more universities. Given the lack of significant numbers of 
matching discipline and/or program HEGIS codes at the four-digit level, it 
was decided to convert to the broader^ two-digit HEGIS level to produce and 
display cost data. Data were still gathered and proces^^ed at the four-digit 
level,. even though they would be summarized and reported at the two-digit 
levcL Summarizing data at the two-digit level increased the extent to which 



2. A coti-allocation technique bn%cd on the premise thai support a4:tivitic»> nuv luntnbatc kosfs u> uiher Nup^H>ft 
activuic* before they afc tilocitcd to Hnai cost obiecttvcf 
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TABLE ! 

0 

Summary of B!Olcx;y by Departmhnt- Disciplines 



Purdue Illinois Colorado Stony Brook Kansas Washington 

0400 Biological Sciences 



X X X X X X 

X X 



0401 Biology 

0402 Botany 

0404 Plant Pathology X 

0407 Zloology 

0408 Pathology 

0409 Phannacy & Toxicology 

0410 Physiology 

04 1 1 Microbiology 

0414 Biochemistry XX X 

0415 Radiation Biophysics 

0416 M.CD. Biology X 

0417 Cellular & Comparative ^ 

0418 Marine Environmental Studies ^ ^ 



X X 

X 

X X 
X 

X XX 

X ^ 
X 



0420 Environmental Studies 

0421 Entomoiogy X X 

0422 Genetics 

0424 Nutritional Sciences X 

0449 Physiology & Biophysics X 

0470 Botany & Plant Pathology X 

0492 Phannacy & Toxicology X 

0494 Bionuclconics X 

0499 Biological Structure 



X X 



quantitative comparisons of disciplines and programs could be made. Ap- 
proximately 75 percent of all student credit hours were now in HEGIS codes 
common to all pilot-test imiversities. 

'While the two-digit HEGIS data comparisons increased the number of 
apparent discipline and program matches among the pilot-test universities, it 
gave no as^jurance to the study group that the underlying activities were in 
fact compariiblc. Son:c two-digit HEGIS categories in fact included widely 
varying activities for which cost data were not comparable. Therefore the 
study group recommends for future studies the use oi" a modified two-digit 
HEGIS coding that would separate such a^eas as: 

• Education (0800 series) into Education and Physical Fducation 

• Fine Arts (1000 scries) into at least the areas of Music. Art, and Theater 

• Health Professions (1200 series) into the discrete h'^alth professions- 
Medicine, Nursing, Pharmacy, Dentistry, Veterinary Medicine, and 
so forth. 

Although some have proposed that individual courses be mapped to HEGIS 
categories according to their specific course content, the study group disagrees. 
Instead, they bcheve that the results of studies are most useful internally 
when they can be readily mapped to departmental organization or major sub- 
divisions. This decision, however, may ultimately decrease the utility of the 
data for external comparisons. 

Course levels used in lEP were expanded from three (lower division, 
upper division, graduate) to five (lower division, upper division, graduate I 
[master's level], graduate II [doctoral-level courses], and doctoral dissertation). 
Because no standard method exists of reporting the credit-hour equivalency 
of doctoral dissertation, data were collected and analyzed based on the number 
of term registrations for doctoral thesis, A sixth course level for professional 
courses, such as health professions and law, might be useful in future studies. 

Dual-level coiu"ses presented a problem to the study group. They adopted 
a covcntion that courses that were targeted toward more than one student level 
should be designed to the level represented by the modal enrollment. Thus a 
course ttikcn by both master's and doctoral students would be classified as 
Graduate II if more than half were doctoral candidates. In developing student- 
program costs, doctoral-level course costs and dissertation-level costs were 
combined. Dissertation units were each imputed at nine credit hours as an 
arbitrary unit of equivalency. 
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The study group used five student-program levels (lower division, upper 
division, graduate I, graduate II and first professional). For external use, the 
piIot-t<!St group felt that comparison of studeni-program-level costs were 
probablv more val«d than discipline-course-level costs because of institutional 
dilTerences in course-level assignments and in assignments of courses by 
department rather than content. 

The study group examined a histogram in comparing the relative enroll- 
ments of student majors in related disciplines as a technique to determine 
prograni similarities. While the group did not rely heavily on this method, 
they concluded that it may be useful in making detailed program analyses. 
(Interested readers should see the Technical Diary [Topping forthcoming].) 



Modifications to the NCHEMS Costing and 
Data Management Software (CADMS) 

The CADMS software was sufilciently general a^d flexible to support 
the parts of the cost study for which it had been designed. Its use in this study, 
however, was complicated by the four sources of fund^^ind the multiple objects 
of expenditure. These additions, together with the basic complexity of MRUs, 
resulted in large matrices and extensive computing time. 

Furthermore, the use of the multiple fund sources and modified direct 
costs called for addit'onal programs within CADMS and for data summaries 
and reports beyond the report-generating elements of CADMS. In particular, 
both MARK IV programs and the Table Producing Language (TFL) of the 
Bureau of Labor Statistics were needed to provide data in summary form. 

This project found that programs in CADMS lacked sufficient tracking 
and control features. Extensive visual inspection was needed to ensure that 
data had been entered correctly. Many errors were found too late, making it 
necessary to rerun several programs to introduce corrected data. If CADMS 
is to be used extensively by MRUs, these error-testing and control features 
should be added, along with readily available tables for f#ilowing allocation 
patterns in processing modified and full costs, 
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Conclusions Regarding 
Data Comparability 



Conclusions from the present work concerning comparability must be care- 
fully drawn. The validity of a comparative conclusion must be tested along 
two dimensions: the data employed must be technically comparable and, with 
respect to the questions posed, they must be sufficient to support substantive 
comparisons. Technical comparability of data is addressed through such 
questions as: 

• Were the data defined in precisely the same way? 

• Were survey instruments identical and were they administered under 
similar conditions? 

• Were the same data categories used? 

• Were data reported and aggregated using the same procedures? 
Sufficiency for substantive comparability of these results raises such issues as: 

• Arc institutional standards for faculty engagement and student achieve- 
ment comparable? 

• Are courses and programs to be compared across campuses in fact com- 
parable in content? 
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• Arc institutional outcomes in instruction, research, and service com- 
parable? 

• Arc there institutional policies or programmatic interactions at one 
campus that will affect the comparability of its data with those of another 
campus? 

No statistical test can address this last set of questions; their answers can be 
found only by persons exceedingly familiar with the programs being com- 
pared. In the case of the major research universities (MRUs), such persons 
would be the department chairmen, academic deans, and vice-presidents or 
chancellors and their staffs who have administrative responsibility for the 
programs. The study group conducting the pilot test focused almost ex- 
clusively on the technical side of the comparability problem. This decision 
was not intended to diminish the importance of the substantive issues but 
reflected recognition of the fact that before any comparisons could be made, a 
technically consistent data base had to be assembled. Once this had been ac- 
complished, selective comparisons could be made by each institution. 
Chapter 5 discusses the appropriate uses and applications of the Information 
Exchange Procedures (lEP) data set from an MRU perspective. This chapter 
assesses the group's achievements in technical comparability and discusses, 
in the context of the questions posed in chapter 1, the problems thus revealed 
in substantive comparability. 



Question 1; Given instiCutional differences in accounting, organiza* 
tion, and so forth, can data be r-Uccted from a group of MRUs in 
accordance with NCHEMS Information Exchange Procedures under 
controlled pilot-test conditions so that the resulting information is 
reasonably comparable? 

The present work has not accomplished this for either the technical or 
the substantive dimension of comparability. This does not mean that such a 
result cannot be achieved. The study group was inclined to conclude, however, 
that Significantly better success than that achieved here would require a pre- 
designed protocol, collaboratively developed in advance of data collection. 

Limitations on the technical comparability of data were discussed in the 
previous chapter. Following is a summary of factors tending to undermine 
the technical quality of the results and of steps taken to mitigate their effects: 
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• Problem: There were few apparent discipline and program matches 
when comparisons were based on foursiigit HEGiS codes. . 
Response: Aggregation to a modified two-digit HEGIS code improved 
apparent comparability, but without assuring substantive comparability 
in curricular details or programmatic purposes. 

• Problem: Dual-level courses were at first not consistently assigned to 
lEP course levels. 

Response: Modal level of enrollment was adopted for classification.- 
Practices varied so widely, however, that a possible bubstantive issue of 
comparability in thei.e data remains. 

• Problem: Institutional policies varied on assignment of credit for thesis 
or dissertation work. 

Response: Assignment of one unit of dissenation work for each term of 
registration, converted in later stages of analysis to full-time equivalency 
by multiplying by nine student credit hours. 

• Problem: Each insiitulion used a different faculty-activity survey. 
Response: A common set of faculty activities was agreed upon, which 
permitted approximate mapping of each institution's unique instrument 
onto a single base. A certain amount of detail and precision of definition 
were lost in this procedure. \^ 

• Problem: In some instances, faculty-activity anal^es (FAAs) covered 
different time periods. 

Response: Participants derived information from best available data but 
at a significant cost to accuracy. 

• Problem: Populations included in the FAAs varied among the six insti- 
tutions. 

Resportse: Some major deviations by single campuses were retrospectively 
corrected, but significant differences remain in the results. This 
primarily affected the comparisons of student-faculty ratios. 

• Problem: Institutions differed in sources of faculty-activity information 
(self-reporting vs. institutional assignment). 

Response: Introduction of the concept of modified direct effort addressed 
the major incomparability encountered. Again, some detail was lust in 
this procedure. 

• Problem: Policies differed in recognition and costing of departmental 
research. 
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Response: Widely differing levels of faculty activuy from FAA& (5.9 to 
19.0 percent) were uniformly prorated back to instruction, eiimin-iting 
the possibility of costing departmental research a* a separate activity or 



(ICR) varied. 

Response: In the latter stages of analysis, general and ICR funds were 
collapsed. The study group was not satisfied that the results of this step 
were fiiUy capable of substantial comparison. 

Problem: Not all fringe benefits were reflected in institutional financial 
statements. 

Response: Adjustments j^^ere made to include fringe benefits paid by a 
state agency and graduate-tuition ^ers as costs of instruction. 

Problem: Some institutions were part of multicampus systems. 
Response. Services provided to a wampus by a central administration 
were costcd and included, ct sis of services by a participafttng institution 
to another campus were excluded. 

Problem: Institutional practice varied as to what constitutes direct and 
indirect costs. 

Response: A consistent distinction between direct and indirca costs was 
made and applied retroactively to the institutional accounting records. 

Problem: Institutional practices varied as to which activities were treated 
as auxiliary enterprises and as to the comprehensiveness of inclusions 
within the auxiliary-fund group. 

Response: Definitions were developed but inconsistently applied so that 
any comparisons among auxiliary enterprises were questionable. 

Problem: The test year could not be consistently defined for all institu- 
tions because of ausence of summer-school data at one university. 
Response: None. At the one university, support costs for a full year were 
allocated to nine months of instructional activity. 

Problem: lEP lacks a mechanism to reflect regional cost differences. 
Re:ponse: None. Regional differences in cost of living for the six pilot-test 
institutions differed by as much as 25 percent. Moreover, the institutions 
experience different climates and consequently different energy costs. Such 
differences are imponant to recognize before making further comparisons. 




28 



Question 2: Does lEP have the capability to deal with interactions 
among instruction, research, and public-service characteristics of 
MRUs? 

As indicated in chapter 3, this question reflects the weakest conceptual and 
procedural link in lEP. lEP instructs the user to prorate its faculty activities 
on the basis of primary intent. Coupled with funding source, this procedure 
served as the primary basis for allocating faculty costs u instruction, research, 
and public service. Whilc^the task force did not applaud this procedure, they 
knew of no other that could solve the problem, given the faculty-activity data 
available for the study. 



Question 3: Do existing information exchange procedures adequately 
account for dififcrent sources of funding in complex, research-oriented % 
universities? 

No. The original lEP made no attempt to differentiate costs by source of. 
funds. As explained in chapter 3, the MRU study introduced four funding 
sources into the IE? cost study: general funds, indirect-cost-recovery funds, 
restricted funds and auxiliary funds. The distinction that gave the pilot-test 
institutions the greatest problem was that between general and indirect-cost- 
recovery (ICR) funds. L^me institutions recognized ICR as a separate funds 
source within their accounting records and could therefore track those ex- 
penditures separately. Others did not distin^niish ICR funds as a separate 
source and therefore had to prorate a percentage of general funds to reflect 
those expenditures. For purposes of comparability, these two fund categories 
were collapsed in many analyses. Howosr, the task force recommends main- 
taining the four separate fund groups whenever the underlying accounting 
data will support such a level of detail. 
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Question 4: Do existing information exchange pro' cdures adequately 
define direct and indirect costs to permit compai^sons among major 
research universities? 

From an accounting standpoint, the division of osts into direct- and 
indirect-cost pools did not hinder this cust study. The accounting systems of 
the six participating institutions were surprisinglv consistent in ihesr handling 
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of accounting chargebacks. lEP as currently written is relatively explicit in 
the types of costs considered direct and those considered indirect (or support 
costs) Procedures for Determining Historical Full Costs (1977), pp 2.14-2.17. 
Tliis definition was augmented by the study group's further delineation of 
costs (Topping, forthcoming, Step 5). 



Quet tion 5: Do the cosdng procedum recommended by the existing 
iuforu*ation exchange procedures adequately reflect the consump- 
tion of support services in major research universities? 

The study group did not examine this particular question as part of the 
pilot test. To do so would have required collecting extensive data from each 
university to capture and reflect usage patterns for each support service. In 
the case of libraries, for example, circulation records for each university 
library would have to be examined to determine users, to allocate costs to 
those users, and to match costs with the student programs in which users 
were enrolled. Given the magnitude of the problem, this task would have 
been very costly. 

Instead, the pilot group adopted the uniform allocation guidelines of lEP 
with some minor modifications (Topping forthcoming. Step 7). These guide- 
lines should be applied uniformly at each institution to ensure consistency in 
the final results. For the most part, this consistency was achieved at the piiot- 
tesr institutions. 



Question 6: How similar are ma|or research universities to one 
another? To other postsecondary institutions? 

All six pilot-test institutions share a similar mission; to promote the public 
mterests of the university as a place of education, learning, and research. 
Furthermore, each institution has been categorized as a research university 
by the Carnegie Commission on Higher Education. The Commission made 
this designation according to the amount of federal financial support received 
by the institution for research in the science and engineering disciplines as 
well as the number of Ph.D. degrees awarded. In addition, each of the six 
institutions is supported predominantly by public funds. The study group 
agreed at the beginning of the study that the institutions shared enough traits 
to make programmatic comparisons potentially meaningful. 
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However, the six universities ditTcred in many respects. They were 
located in different geographic regions of the country. Two are situated in 
metropolitan areas, the other four in small cities. While their missions Sic 
fundamentally similar, there are important UitTerences. For example, Illinois 
and Purdue are land-grant universities and consequently devote a larger 
share of their resources to agricultural research and cooperative extension. 
Washington and Stony Brook have health science centers located on the 
campus of the parent university, while in three of the four remaining univer- 
sities, the health-related programs are ofTcred on separate campuses. 

in addition to differences related to mission, there were important dif- 
ferences in size, in this study, Illinois, I'urdue, and Washington, which con- 
stituted one comparison group, had student bodies, faculties, and budgets 
roughlv two to three times as large as those of the other three institutions. All 
three are comprenensive univcr^ties oiTering a full range of student programs 
at all student levels. Sponsored research activities accounted for nearly one- 
fourth of their total eJi^^enditures. 

The Universities of Colorado and Kansas, which emerged as a second 
comparison group, had student bodies, faculties, and budgets of similar size. 
Neither is the land-grant institution m its state, and both emphasize similar 
disciplines and programs. Sponsored research, while a major component (on the 
average, 17 percent for 1975-76) of each university, was not of the magnitude 
of that of Illinois, I'urdue, and Washington. Programmatic profiles of these 
two universities dilTered significantly from those of the larger three. 

Stony Brook, the most recently\established and smallest of the six univer- 
sities, constituted a special case. Its instruction program was about one-third 
the size of that of Purdue, Illinois, or Washington and roughly half that o. 
Colorado or Kansas, based on the number of student credit hours taughi. 
Stony Brook emphai^izes programs in engineering, mathematics, and th ■ 
physical sciences (high-cost disciplines), particularly at the graduate level, an J 
does not olTer graduate work in business and education (low-cost disciplines) 
Although the research component at Stony Brook was relatively large (19 p<^i 
cent of" total expenditures for 1975-76), it was restricted mainly to engineering 
and the physical sciences. Located in the New York metropolitan area, Stony 
Brook operated at a cost-of-living exceeding that of the other five universities. 

Given these technical difTiculties associated with IHPand these ditVercnces 
among even apparently similar institutions, what can be said regarding com- 
parability of data-' 
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• Hierarchies of unit effon, or cost by level of instmction and discipline 
within one campus, were reascnably consistent with those of the other 
institutions. Overall cost ratios by level of instruction were 1.0 (low^r 
division), 2.0 (upper division) and 4.0 (graduate). High-cost discip?i • 
(such as engineering or the physical sciences) tended to be about ^ 
and a half times more costly than less expensive ones (such as psychoiog}^ 
and other social sciences), though this varied by institution and level of 
instruction. The traditional assertion of MRUs was supported: no 
meaningful assessment of resource requirements can be made without 
attention to these differences. 

• Variability on a particular level and discipline mdex, in terms of either 
cost or effort, was significant across the campuses and nicreased with level 
of course or student. Cost variability among institutions was approxi- 
mately ±20 percent of the mean at undergraduate levels and approxi- 
matcly ±40 percent of the mean at graduate ones. It is unclear whether 
this was due to differing faculty-activity analyses, differing allocation 
strategics, or a combination of both. 

• No conclusions can be made regarding sponsored research, public service, 
and their relationship to instructiui;. 

• Apparent coherence of data increased when analyses were performed of 
student-program rather than discipline costs. This finding suggest^ ^hat 
student-level designations were more consistent across the six institutions 
than were course-level designations. 

• Within each institution, full unit costs were highly correlated with modi- 
fiei direct costs, across student programs and disciplinjes, and across 
strident and course levels. (Correlation coeffivients often exceeded .95.) 
For interinstitutional comparisons, full costs may be more comparable 
than modified direct costs because institutional differences arising from 
accounting practices, organizational structures, and funding levels for 
indirect -cost centers were minimized during the latter steps of the cost in 
procedui'es. 

.Throughout the pilot test, the MRUs encountered many problems that 
impaired technical comparability of their data. Adjustments were made for 
some of these problems, but others could not be easily corrected, in part, 
pilot institutions were handicapped by having to conduct a cost study after 
the fact, particularly with the faculty-activity data. In addition, the study 
group feif that the protocol statements should be more complete and more 
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fully discussed by che pariicipanis before collecting any data. Start" at the 
participating institutions stated that costly data re-runs could have been 
avoided if they had been better informed at |he beginning of each procedure. 
Finally, the study group concluded that the study could have been conducted 
more efficiently if the time frame had been shortened. MRlTs attempting to 
replicate this study should not expect to produce technically c parable data 
during the initial year, which should be viewed as a test year. ^ the second 
and third years, howevexj^ attempts at producing cost data among a group of 
similar institutions should increase the vali'^-^ ' of those data. 
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Concluding Remarks and 
Institutional Observations 



The six institutions involved in this study undertook to lest the applicability 
of ' he Information Exchange Procedures (lEP) in the major research univer- 
sities (MRUs). Throughout the study, conceptual' and technical problems 
were encountered. Tnese were addressed and, in some cases, procedures were 
modified to reflect the complexities of the MRU. 

What are the strengths and weaknesses of lEP from an MRU perspective? 
First of all, lEP constitutes a set of well-developed, well-documented pro- 
cedures for performing cost analysis at both the full- and direct-cost levels, 
with emphasis upon instruction. Developed under the direction of a national 
task force, lEP was pilot tested in 60 institutions of varying size and type 
before being released for general use. In addition, lEP has associated computer 
software to support the development of discipline and program-unit costs. 

lEP facilitates the aevelopment of unit costs of instruction that can be 
used in comparing not only entire institutions but also individual programs. 
These procedures enable one to summarize data at several levels and at dif- 
ferent degrees of detail. Finally, lEP is the most widely publicized and perhaps 
the most commonly used costing model in the country. Since its introduction, 
several hundred institutions have attempted to implement it. 
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Nevertheless, lEP is extremely cumbersome, requiring significant ^tafT 
time to conduct a cost study. lEP requires substantial levels of detail, regard- 
less of whether one is attempting to study an entire institution or a small 
section of it. lEP spreads costs to basically three cost objectives (instruction^ 
research, and public service), as opposed to a larger number of cost objectives 
that may be of more importance to mstitutional administrators and regulatory 
agencies* Because IE? relies on a generalized program structuie, administra- 
tors may have difTsculty reconciling final cost data to their panicular organi- 
zational and administrative structures. Moreover, lEP relies heavily on a 
faculty-activity analysis to spread costs to these three cost objectives. As this 
study has demonstrated, faculty-activity surveys must be designed and con- 
ducted in precisely the same way to produce comparable cost information. In 
many instances, this will be an impossible condition to meet. Prospective 
participants in a cost study should realize at the outset that failure to meet 
this condition will substantially weaken the final results. 

The question of coijjparabiiity of cost data was of major concern to the 
paat.cipating institutions. At the conclusion of each step of the costing process, 
time-consuming review procedures were followed. At each point, the study 
group decicJed'tFial^while differences did exist among the six data sets, the^-^ 
differences were not fundamental enough to halt the pilot test. At the end of 
the pilot test, the group concluded that even though the data were neither 
substantively nor technically comp rable, significant strides had been made 
toward achieving technical comparability. The group also thought that with 
repeated applications and with well-defined protocols written and ap-eed to in 
advance of data collection, the chances of producing comparable data sets 
would be significantly improved. 

But achieving technical comparability is only half the battle. IKP contains 
no mechanism capable of assessing the similaritie*^ or dissimilarities of the 
underlying activities being compared. The study group referred to this di- 
mension as substantive comparability and agreed in principle that a review 
of underlying program activities was vital to any successful information 
exchange. The pilot-test group did not develop any procedures or protocols 
for accomplishing this task, However, the other subgroup of the MRU task 
force addressed this question and developed a prototypical program statement 
that examines the goals, objectives, and curricula of programs being com- 
pared, In this area, the two subgroups, while working independently, begin to 
reinforce each other. Future exchange ctTorts among MRl'S should examine 
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the work of both subgroups to better understand the complexities of infor- 
mation exchange. 

The study group discussed the potential uses of cost data from an MRU 
perspective. In general, lEP was designed to highlight cost difTerences among 
academic disciplines and student programs. It is retrospective in that it ex- 
amines data from a past period, usually the last academic or fiscal year. Its 
applications, howe%'er, are often prospective, since past cost behavior is often 
used as a criterion for justifying existing levels or changing resource-allocation 
patterns in successive fiscal periods. Because higher education lacks weli- 
defined, measurable outcomes, cost data from comparable programs and 
institutions often serve as guidelines in determining budgetary allocations. 

Various uses for cost data can be postulated for MRUs. For example, 
department chairmen and academic administrators could analyze course- 
taking and resource-utilization paaerns. Institutional administrators may 
find cost data useful in studying resource-distribution patterns among depart- 
ments and disciplines as well as in reviewing curricular and degree patterns. 
The use of cost data by state agencies external to the university presents a 
conflict for the MRU. MRUs have an interest in presenting cost data at a 
sutTiciem level of detail so that programmatic difierenccs and their associated 
costs are clearly evident. However, MRUs often fear that external agencies 
will use detailed information to enact accountability and control procedures 
af a similarly detailed level. Nevertheless, it is clear that many state agencies 
have a need for and can use cost, resource, and student information in funding, 
program-review, and formula-development projects. The appropriateness ot 
collecting and analyzing specific types of information, including cost infor- 
mation, would depend on the context and process for decisionmaking within 
each state. 

This report should not be interpreted as a blanket endorsement of 11! I' 
for use within an MRU environment. Like other large-scale costing models, 
lEP still has both conceptual and technical problems. Nevertheless, the pilot 
test described in this study made significant strides in correcting many of 
them, MRUs recognize the role that cost information often plays in resourcc- 
:' allocation decisions— both within the university and in governmental agencies, 

; Given this role, it is important for MRUs to have the capability of producing 

/ timely and valid cost information. It is in this spirit that fhe study was under- 

taken and its recommendations oOered. 
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institutionai Observations 

The author intended to conclude this document with a discussion of the 
potcnaal applications of cost data from an MRU perspective. However, he 
quickly discovered that not all universities are of one mind on this subject. 
Some representatives of the pilot-test institutions stressed the utility of the 
data for internal management but warned against potential misuse of the data 
for exiemai comparisons. Others argued that the only real use of the data was 
to be made bv external agencies in allocating funds among institutions. Some 
contended that lEP as modified by MRUs was still deficient in many resf>ects 
and should not be relied upon for cost comparisons among MRUs, while 
others were willing to accept lEP data as reasonable proxies for overall costs. 
In short, there was no consensus among the pilot-test institutions on this 
important subject. To resolve this dilemma, we asked each of the pilot-test 
institutions to draft a statement outlining its experiences with lEP and dis- 
cussing potential applications of cost data from an MRU perspective. Their 
statements follow verbatim. 
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University of Washington and University of Colorado 
Joint Statement Regarding 
Potential Uses/ Applications of lEP in MRUs 

We are very concerned that the higher ec*ucation community view the 
lEP project in its totality, that the reports of both subgroups of the task force 
be considered as elements of the same study. We believe thaf reports of the 
two groups have many significant areas of agreement, that the extent of this 
agreement was not fully realized until the pilot test was complete, and that 
these philosophical as well as procedural overlaps should be summarized. 
We believe thut the two approaches and their apparent "coming together" 
indicate that cost analysis for MRU's should be carried on in the context of 
both reports, for only in this way can both substantive and technical com- 
parability be addressed. 

As the EA&A subgroup members became more familiar with each other 
and with the lEP procedures, there was a growing sense of unease about tlje 
applicability of lEP to major research universities, even though the group 
agreed to go forw^^d at each individual step in the process. While members 
of the group ostensibly represented similar institutions, sufficient significant 
differences were disclosed that the application of lEP-MRU data to any real 
world policy problems is not advised. 

The following points represent our view of the potential application of 

lEP data: 

/-Unmodified IHP is not suitable for MRUs 

-With substantial modifications of the kind performed in the pilot test, 
lEP may provide a beginning place for internal analysis 

— Significantly improved technical and substantive comparability will re- 
quire predesigned protocols for collecting accounting and expenditure 
dafa as well as uniform FAA instruments 

— lEP cannot be unqualifiedly endorsed for application across various 
kinds of institutions 

We feel that another key to obtaining agreement about the validity of 
cross-institutional information is not onlv to have predesigned instruments 
but to have agreement on and understanding of the potential applications ot 
the c Jta, 
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In general the lEP approach, adrnirable and methodologically correct as 
it may be, is so expensive, complicated and fraught with potential error that 
we believe simpler approaches may be more appropriate guides to public 
policy in this area. 
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University of Uiinois 
Potential Uses/ Applications of lEP in MRUs 

The four cells in the following matrix provide a perspective for an evalu- 
ation of the p^^tential uses/applications of lEP in MRUs. 



"^^Data"''""--^,.^.^^^ Users 






Base ^""^^^^"---.^.^^^^ 


Internal 


External 


Sing!e MRU 


(1) 


(2) 


Multiple MRUs 


(3) 


(4) 



The columns of the matrix represent internal (institutional) or external 
(funding body or coordinating agency) users of data produced by lEP. The 
rows represent the lEP data base, i.e., a single MRU or multiple MRUs. 

Cell(l), Institutional User— Single MRU. Since each of the six task force 
institutions applied lEP to its own data, there is no question that IE? can be 
used for institutional self-study by MRUs. The University of Illinois found 
the IE? :,oftware to be reasonably easy to use; and, to produce information 
similar to that resulting from internally developed software. The potential 
uses/applications of lEP by MRUs for internal purposes can be answered 
individually by each MRU by comparing the costs and benefits of using the 
lEP software versus those of developing their own. On balance, it is likely 
that those institutions that have invested heavily in their own programs to 
produce IWLMs, unit costs, and so on will maintain those programs, while 
institutions that have not developed t leir own software might find lEP to be 
attractive. 

Cell (2). External User-Single MRU. An MRU might find lEP to be 
useful to produce information for external agencies in the same manner as for 
its internal purposes. However, external us*; of information about a single 
MRU, or any other single institution, is marginal, other than for the establish- 
ment of trends for that institution. Note that we have precluded a judgment 
of the combination of lEP information for MRUs with that of non-MRUs in 
this cell, since this task foice only contained MRUs. 

Cell (3). Internal User-Multiple MRUs. MRUs might choose to ex- 
change information produced by lEP for their internal use. When evaluating 
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the application of lEP for intcr-institutional data exchange, we found il useful 
to further segment this cell as follows: 



Campus- or System-Level Administration 



a) Internal Use 



c) External Use 



College or Departmental Administration 
b) Internal Use 



a) Campus- or System-Levei Administrafion—Internal Use and b) College or 
Departmental Administration— Internal Use. Inter-institutional data exchange 
among MRUs of lEP-produced information probably would focus on such 
items as productivity ratios and modified direct or full costs by discipline and 
program and course or student level. The task force did not address the question 
of substantive comparability, and lEP by its very nature cannot address that 
question. Thus, we feel that the techniques suggested by the Measures and 
Definitions Subgroup are more appropriate for inter-institutional data exchange 
i/thc intended use of the data is internal, e,g., to assist central administrators 
in budgeting or college deans in program evaluation* 

c) Campus- or System-Lei^el Administration— External Use. Central admin- 
istrators of MRUs, including the University of Illinois, have found the ex- 
change of certain items of information to be of value, e.g., the annual exchange 
of faculty salaries by rank and discipline. When using such information, it is 
clearly understood that each and every discipline is not substantively com- 
parable between the respective institutions. Nonetheless, the judicious appli- 
cation of the aggregate data, e.g., by eliminating salaries of clinical facuhy 
for some purposes, has been found useful for reaching some **over-air' conclu- 
sions concerning the level of salaries at a given institution vis-a-vis its peers. 

The University of Illinois feels that a oimplified or reduced data base 
(versus that used in the EA&A study) produced by lEP could be usefully ex- 
changed by MRUs to help them assess **overair' levels of funding, par- 
ticularly with respect to their instructional programs. This reduced data base 
would be limited to one fund source {eithfr General or the combination of 
General and ICR funds), i.e., restricted and auxiliary funds would be 
eliminated, and would contain modified direct and full costs of instruction 
by 2-digit HEGIS discipline or student program and course or student level 
as suggested by the task force, along with the credit hours in each of those 
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cells. This level of aggregation appears to be sufficient to "control" for basic 
differences in the magnitude and mix of instructional prtigrams between 
MRUs, without becoming enmeshed in an evaluation of substantive com- 
parability for each program. Obviously, any such information exchange 
would nave to be baied on a set of data that was reasonably comparable 
technically, particularly with .espect to the HEGIS mappings and financial 
protocols. This level of technical comparability appears to be achievable. 
While it is less likely that technical comparability can be achieved in the 
Faculty Activity Analysis (FAA), the use of the aforementioned reduced set 
of activities can enhance that likelihood. Moreover, the modified direct costs 
method suggested by the task force minimizes the need for precise technical 
comparability of FAA's, especially when the focus is aimed at an assessment 
of overall funding levels. 

Cell (4). External User— Multiple MRUs. The University of Illinois views 
this cell to be similar to Cell (3-c) discussed above, in the same context as it is 
with the exchange of facuhy salaries among MRUs. That is, MRUs would 
engage in inter-institutional (lEP) dsta exchange and would provide certain 
results of that data exchange to external agencies following a protocol agreed 
upon by the exchanging institutions. 

Summary. The University of Illinois' position on the appHcation/use of 
lEP for MRUs is as follows: 

1. lEP's internal use for institutional self-study can be evaluated by each 
MRU on purely cost/technical grounds. Based upon such evaluations, 
the MRUs might choose to implement all or portions of the lEP 
software. 

2. IE? docs not support detailed internal program or budget analyses based 
uf>on inter-institutional comparisons because it does not address the 
issues related to substantive comparability. The techniques proposed by 
the Measures and Definitions Subgroup appear to be better suited to 
such analyses. 

3. l^P could support the exchange of data between MRUs at a level of 
detail described above, where the objective of such data exchange is to 
assess overall instructional funding levels in a manner similar to the com- 
parison of overall faculty salaries. The University of Illinois encourages 
the development of such exchange groups and would be willing to 
participate in them. 
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Uciversity of iCaniias 
Potential Uses/ Applications of lEP in MRUji 

The University of Kansas generally supports the concluding remarks of 
Chapter 5 based on its c:sperience as a member of the Experimental Applica- 
tion and Analysis Group. Hov^ever, several comments relevant to the use of 
lEP in a major research university environment are necessary. 

While there is little question that there are significant structural differ- 
ences betu'een major research universities and nonmajor research universities, 
there are also significant differences between MRUs. Tnese differences must 
be controlled for when an MRU chooses peers for comparison purposes. 
Within the task force there were several distinct differences among participa- 
ting MRUs. Institutional size and mission was one obvious factor. Another 
was the co-mingling of land grant institutions, noniand^rant institutions 
and institutions which specialized in a limited number of well supported 
programs. The results of the MRU-IEP project may well have been more 
meaningful had the comparisons accounted for these variations. Many of the 
cost comparison differences can be attributed in pan ro the lack of institu- 
tional uniformity and may have helped to obscure lEP's capability to support 
a meaningful comparative analysis among MRUs, 

The* project was also limited by the task force's inability to perform 
compara^ ve analysis in the research and public service sectors. Part of this 
limitation us caused by the confounding differences resuhing from the 
research and public s^ervice mission in a land grant university as compared to 
a j^eneral purpose stale university. Moreover, the task force was not able to 
sufliciently define common denominators for public service and for research 
which would make these costs as meaningful a^ instruction has become for 
institutional comparison purposes. This Hmitation creates an overemphasis 
upon instruction as the final cost objective in MRUs, thus raising serious 
questions concerning the use of lEP as an appropriate cost analysis tool for 
MRUs. While we recognize that there are limitations in any cost study, we 
are concerned that the above problems not be overlooked by zealous users of 
cost comparison data. Our ability to cost instruction in such detail, while not 
having comparable analytical support for research or public service, suggests 
that instruction is the primary factor in MRU comparisons. This may in- 
correctly imply to the lay person that instruction is the sole measure of insti- 
tutional performance in MRUs. 
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Finally, there is a critical need in MRU-IEP for more emphasis in costing 
the support areas, specifically in library, academic administration, student 
services, institutional support and physical plant. Funding agencies as well 
as university administrators may well desire to compare expenditures in 
these areas as well as those related to the institution's academic mission. The 
ability to develop measures of comparability among institutions for these 
supfwrt areas could result in much more meaningful and usable studies. 

In summary, the University of Kansas believes that appropriately modi- 
fied, lEP and similar cost analysis techniques can be used effectively for 
comparative analyses across major research universities. By controlling for 
dilTerences such as institutional size and mission, more useful comparisons 
can be made. 
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Purdue University 
Potentiai Uses/ Applications of lEP in MRUs 

Various cost studies and data exchanges have prevailed in the higher 
education community for many years. Perhaps the simplest and most fre- 
quently requested information relates to salaries, both individual by position 
or averages. 

These data exchanges found a receptive audience in institutional admin- 
istrators^ as they examined their own institutions and presented their case to 
external agencies. Similarly state officials and legislative groups found them 
helpful in carrying out their allocative decisions process. Use of data by insti- 
tutions, state agencies and legislative groups is almost certain to continue. 

For internal purposes, comparing departments, programs, cost, etc., or for 
a single institution's external purposes, the Information Exchange Procedures 
(lEP) provide a complete set of easily adaptable computer programs. Although 
these procedures can be more easily run in smaller, less complex universities, 
they also can be run at the larger more complex institutions participating in 
this project. 

What then, are the problems confronting major research universities 
(MRUs), state agencies, or legislative groups using lEP for providing data 
input purposes? 

To answer this question these procedures muA be broken dowr m^u 
their various components: student data, personnel data and financial ilata. 

Given a reasonable student record system the student data module is a 
very useful tool in analyzing courses taken by student majors, on the avt r^^gc, 
and to whom departments taught courses. This module is equally successful 
for large complex universities and smaller less complex schools. 

The problems encountered in this specific project did not revolve 
around the systems designed to carry out these tracking processes, but rather 
the differences in student program labels, course levels, departments and the 
consistent assignment of credit hours, particularly at the graduate level. 
With some amount of effort to assig;n these labels consistently there should 
not be problems with comparability or accuracy resulting from this part of 
the exchange procedures. 

As in any cost study Purdue University has been involved with, the most 
diflucult part of the information excnange procedures was the assignment of 
faculty effort, or the so called personnel data. 

If an institution, agency, etc. feels a great need to identify all faculty 
effort into detrJleU conip< is: undcrgiaduate or graduate instruction, public 
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service, non-sponsored res»>arch, student counseling, professional develop- 
ment, etc.— major research mstitutions will always have ditTiculties as they 
will with lEP. A potential -olution to these difikulties wor.id be to predeter- 
mine detailed definitions of the various activities which carried out and 
tnen design a system to collect (his information. It is questionable whefhci 
even this process would result in greater comparability. 

If on the other hand, the audience for whom the study is lieing conduced 
is willing to accept that the mission of an MRU is to carry out non-sponsored 
research, public service, graduate instruction, professional development, etc. 
and that these are all necessary parts of the instructional function, the detailed 
reporting of these activities is eliminated along with the definitional problems 
associated with them. 

At Purdue University and in the multi-institutional studies conducted in 
(he State of Indiana, it has been accepted that all of these non-sponsored ac- 
tivities earned on by our faculties are part of the instructional prugrams wr 
otler. Thus, we have elimin-ited the necessity to cost these activities scpa- 
rateiv (separately budgeted activities such as AHS, CHS, etc. are excluded), 
Admittedly this decision probably increases fhe cost of instruction, it Oiu- 
could separate these functions from pure instructional activities or } . . -ynied 
that they are not part of the instructional mission. This piocedure allov.s liu- 
ma)ontv of cost to be assigned to the most commonly ident!»";cd uiiit ot 
student programs, and/or credit hours. 

The problems encountered by the MRl^s m this project primarily resiiln u 
Ironi procedures contained in the personnel portion of IHl' c iihng lor ilic 
spectiic idenrification of these various activities and our inability to dcvi-Ioji 
definitions for them which would allow the identifications called tor. 

While the MRU project did aggregate some functions, there was sufli 
uent concern about definitions used among the institutions \o canst- \hv 
lesuitmg information to be suspect. 

The third and final step in the e.xchange proccdui.-s is the su >.;i!ird 
jccount crossover module which brings the financial data into the proper 
jcu'unis lot costing. Original Il-T procedures, which were quickly discarded- 
nioved the dollars around without ct)nsideration of source of funds, in sudi a 
wav as to make them completely useless for an institution accepting and 
spendini! so called restricted dollars. Once this problem is solved it is still 
ncLCssan. m niulti-institutional comparisons, to e.Kannne careiullv the a>. 
eountMig and telated inanauenient practices of the institutions It) asceitaiii 
telaiive lonsisten^x m the fuiaiu ial d.ila befoie the (.tossoNei prtkeduies :iu' 
empi" '\'ed 
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Again, if the audience is willing to look ai all cost as full cost of intstruc-^ 
tioiij many of these problems arc eliminated. It then makes no ditference if 
computer, library, physical plant, etc. cost is direct charged or aggregated 
into cost centers with the charges layed back on some reasonable basis. 

The above descriptions point out some of the problems encountered in 
employing the NCHEMS Informanon Exchange Procedures at major 
^esearcli universities and presumably at non-major research institutions. 
They should not however, lead to the conclusion that exchanges of the type 
of information produced by lEP are impossible. They are possible but must 
b*: accomplished through carefully controlled and examined steps, focusing 
on what the desired outputs and uses are. 

With appropriate etfort IHI^ can be moditicd to serve these purposes as 
has been done by the publicly-supported institutions in the S?;itc of Indiana. 
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State University of New York at Stony Brook 
Potential Uses/ Applications of lEP in MRUs 

Stony Brook has welcomed the opportunity to pariicipaic in both the 
Experimental Application and Analysis and the Measures and Definitions 
subgroups of the NC:HP.MS Task loix^. \X c have benefited greatly from 
both. 

The Measures and Definitions report is of fundamental signilkance in 
establishing means by which MRUs might be substantially compared, taking 
adequate account of their inherent comf lexity and of essential academic dil- 
ferences in their superficially similar programs. Its application as a single 
campus exercise produces important results in improved understanding oi 
mission, status, and planning quite independent of its utility for mtennstitu- 
tional comparisons. It is unfortunate that MRl's hiive not had the foresight 
to press this form of assessment previously. 

Even though measures like IHP permit at be:.r only a primitive represeri 
tation of an MRl\ the work of the Experimental Application and Analysis 
group has been extremely important as well. 

• lEP is widely applied by reviewers of higher education. Any MRl' that 
functions within a diversified system or that draws significant resources 
from multipurpose national programs ol" support to higher education 
should expect that such measures will be applied to it. The pn ^ability 
IS high that such comparisons will result from the reasonable insistence 
of public decisionmakers. Alfected MRUs should be prepared to assist 
in that process; preierabiy, they should supply leadership in advance ot 
need in exploring its pitfalls, 

• Even if a history of results from the Measures and Definitions procedures 
were in hand, it would still be necessary to report some results with 
procedures similar to lEF. That repon could then be qualified by more 
sophisticated analyses of substantive diBerences not uncovered m the 
lE.F format, bui the generalization of lEP developed here would still be 
necessary to meet the insistent demands of external decisionmakers 
We at MRUs who are accountable to multiple constituents should be 
prepared to map our responses to questions into the language that wc 
know must be available to our reviewers or should expect otht-rs who 
have less insight into our programs lo do mi tor us. 

• It IS correct that technical comparability between one MRU and a selected 
luimhci of other .MRUs c;tn be inor-.- rapidly and cconornicaUy obtaint-d 
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by mapping the others onto the single MRU lhan by mapping all mio a 
single, generalized structure. However, this technique applied more than 
a few times in partii^ar cases leadS"t"?J^holarship beyond our groves. 
' The importance of the basic assiunpMX^i that we in the Experimental 
Application and Analysis group set out to test in our generalization of 
lEP— that there exists a singl^^i^sufTicient frame^¥cjrk to sustain technical 
compiarisons of a set of Ml(tJs— p^sists in spite^f the availability of 
this option for quicker resultf. ^, 

• We say candidly however, thai there (probably ^ no MRU that can in- . 
ternaily benefit by mapping its custoi^ary measures of resources and ^ 
products into a generalized lEP. The extra effort required for a con- / 
sistent representation is entirely to respond to demands for externaP^ 
comparisons, 

• While we have been at this work another urgency has been added to our \ 
task: the impending federal requirement for information concerning full 
effort of faculty engaged in sponsored research will doubtless be followed 
by the insistence of state agencies that they receive the same data about 
their research-intensive campuses. 

We agree with much of the commentary from the other participants con- 
cerning how the generalization of lEP to MRUs might be improved^ but 
would add these points: 

• Comparisons of direct and modified direct indices should concentrate 
equally on faculty effort and cost measures. It is not a weakness of lEP 
that the analyses of the pilot-test institutions were predominantly on 
costs; that resulted rather from pressures of time. However, this concen- 
tration on cost measures yielded fmal data which were systematically 
incomparable to the 20 percent level because of cost-of-living and conse- 
quent salary differences at one institution. 

• Stony Brook regrets that the choice of a retrospective approach led to data 
which precluded an attempt to address the question of joint products; 
MRUs will now have to address that question individually. 

• Stony Brook does not agree that total unit-cost results are of technically 
higher quality than modified direct costs. The additional consistency 
Which may result from combining the costs of institutional support ac- 
tivities with instruction is bought at the price of inclus'.on of addition^il 
data which may introduce very substantial differences, either because of 
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differing institutional practices or because of substantial regional cost 
differences (e.g., utilities). 
• Indeed, Stony Brook questions the usefulness of the total cost results of 
lEP for MRUs, and urges that service and support costs be measured 
only by function and with the same attention to differences in policy 
and practice which !EP bestows on instruction. In discussions between 
MRUs, no significance can be assigned to the single difference, total less 
direct costs. We should encourage our reviewers to examine primitive 
data in this area as consistently as we encourage them to consider instruc- 
> tional data which have been sufficiently disaggregated to show di^rences 

/ by instructional levei and discipline. f 

^ .» 

Stony Brook would welcome opportunities to repeat the Experimental 
AppUcation and Analysis drill, applying the lessons of this exercise to improve 
technical comparability. We would also welcome an opportunity to test with 
other MRUs the Measures and Definitions instrument. 
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